moq-mux: decouple importers from the catalog, split byte-parsing into per-codec splitters, and make importers pure frame publishers#1749
Merged
Conversation
First slice of the import refactor. The opus importer no longer holds a `catalog::Producer` and mutates a shared catalog with a `Drop` hook. It now produces frames on a single track and exposes its own standalone `hang::Catalog` rendition, which a new `publish` bridge merges into a broadcast catalog (removing it on drop). - `codec::opus::Import`: `new(TrackRequest, Config)` (on-demand) and `from_track(TrackProducer, Config)` (broadcast push / fixed track). `decode` now takes `impl IntoIterator<Item = Frame>`; `decode_buf` is the raw-packet convenience that stamps a wall clock when no timestamp is given. `catalog()` returns the standalone `hang::Catalog`. - `publish` module: `Renditions` trait, `Published<I>` (merges renditions into a `catalog::Producer`, retires on drop, derefs to the importer), and `unique_track` (mints a legacy single-codec track at the microsecond timescale). - Rewire `import::Framed`, moq-gst, and moq-rtc through the bridge. Tests cover both construction paths (TrackRequest and broadcast unique_track), frame delivery, the catalog merge, and retire-on-drop. https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Second slice of the import refactor. The H.264 importer joins opus on the
request-based core: it no longer holds a `catalog::Producer` or mutates a shared
catalog from a `Drop` hook. It produces frames on a single track and exposes its
own `hang::Catalog`, which the `publish` bridge mirrors into a broadcast catalog.
Unlike opus, H.264's catalog is lazy (avcC for avc1, the first SPS for avc3) and
refines over time (jitter), so `Published` gains a `sync()` that re-mirrors the
importer's renditions and is a no-op when nothing changed. Callers invoke it
after each decode.
- `codec::h264::Import`: drop the `E`/`catalog::Producer`/`TrackProvider`
coupling. `new(TrackRequest)` (on-demand) and `from_track(TrackProducer)`
(broadcast push / fixed track), `with_mode`, lazy `catalog() -> Option`,
`Renditions` impl. avc1 reconfiguration now errors instead of minting a new
track (a single fixed track can't represent a new init segment); avc3 SPS
changes still update the rendition in place.
- `publish::Published`: generic over the catalog extension `E` (so it attaches
to a container's extended catalog), plus `sync()` for lazy/updating catalogs.
- The TS container builds its per-PID H.264 stream through `Published`
(`unique_track` + `from_track`), syncing after each decode; external behavior
is unchanged (byte-exact roundtrip tests guard it).
- Rewire `import::{Framed,Stream}`, moq-cli, moq-video, moq-rtc through the
bridge. Drop the now-unused `TrackProvider::set_suffix`.
Also fixes a pre-existing missing `.await` in the moq-ffi tests
(`dynamic_track_request_can_publish_media`) that broke `cargo check
--all-targets` on dev, unrelated to this change.
https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
…Option moq-video's `Producer::track()` now returns `&TrackProducer` (the avc3 track is always created eagerly), so the `.expect(...)` no longer applies. This broke `cargo check --all-targets` on moq-boy. https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Completes the import reshape: h265, av1, vp8, vp9, aac, and the legacy
audio importer (MP2/AC-3/E-AC-3) now follow the same request-based core as
opus/h264. None hold a `catalog::Producer` or mutate a shared catalog from a
`Drop` hook; each produces frames on a single track and reports its own
`hang::Catalog`, attached to a broadcast catalog through `publish::Published`.
- Each importer: `new(TrackRequest)` / `from_track(TrackProducer)`, a local
catalog, lazy `catalog() -> Option` for the video codecs (config known on the
first key frame / SPS) and eager `catalog() -> &hang::Catalog` for aac.
Reconfiguration on the fixed track is an error (no new-track minting).
- `import::{Framed,Stream}`: every codec arm mints via `unique_track` +
`from_track` + `Published`, syncing the video codecs after each decode.
- TS container: H.265, AAC, and legacy per-PID streams build through
`Published`. AAC's synthesized `description` and audio-burst `jitter` are set
via a `pub(crate) aac::Import::rendition_mut` + `sync`, since the rendition now
lives in the importer's local catalog. External behavior is unchanged (the
byte-exact roundtrip tests guard it).
- Delete the now-unused `TrackProvider` (every codec mints via `unique_track`).
All codec/TS importers now share one shape; `TrackRequest` is the on-demand
entry point and `from_track` the broadcast-push one.
https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
First slice of the splitter / unified-decode work. - New `codec::h264::Split`: a standalone parser that turns H.264 bytes into `container::Frame`s plus a resolved `VideoConfig` (the NAL/AU assembly, avcC parsing, SPS/PPS cache, and Annex-B wall-clock all live here, independently testable). avc1 still errors on reconfiguration; avc3 still updates in place. - `codec::h264::Import` now drives `Split` internally for its byte APIs (`decode_frame` / `decode_stream` unchanged, so the Framed/Stream/TS dispatchers and external callers are untouched and don't regress) and pulls the resolved config into its local catalog. - New `publish::FrameDecode` trait (`decode(impl IntoIterator<Item = Frame>)`), the uniform decode entry point a caller drives with frames from its own `Split`. `Published::decode` wraps it and syncs, so the catalog re-mirror can't be forgotten — the footgun-free path. h264::Import implements it. The dispatchers still use the byte API + manual `sync()`; migrating them to `Split` + `Published::decode` (and resolving the avc1-vs-avc3 config flow on that path) is the next slice, then rolling the same split to the other codecs. https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Per review: the splitter shouldn't know what a codec config is. It just takes a single byte stream (no out-of-band init), finds access-unit boundaries, and packages SPS/PPS into each keyframe so every keyframe is self-contained. - `codec::h264::Split` is now an Annex-B stream assembler only: no avc1, no `VideoConfig`, no `take_config`. It exposes `decode_stream` (unknown boundaries), `decode_frame` (one access unit), `decode_from`, `seed` (prime the SPS/PPS cache from an out-of-band parameter-set buffer), and `reset`. Wall-clock timestamps for stdin live here. - `codec::h264::Import` owns all config, from exactly two sources: an avcC handed to `initialize` (avc1, required), or the SPS the splitter packages into the first keyframe, scanned out of the frame here (avc3, no init needed). It also owns the avc1 length-prefixed framed path; the stream splitter is Annex-B/avc3 only, matching "if you can init out of band you already know frame boundaries, so you don't need the stream splitter". - A keyframe that can't be configured (no inline SPS, no avcC/seed) is a hard error. A non-keyframe before the first config is tolerated: it's a mid-stream-join leftover that the producer's lenient start drops ahead of the first keyframe (preserves `survives_midstream_join` and the dirty TS joins). https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
The lazy-catalog codecs all did `decode_frame(...)?; sync();` by hand, an easy-to-forget two-step. Make `Published` own the pairing so the catalog re-mirror can't be skipped. - New `Published::decoding(|inner| ...)`: runs a decode/edit on the inner importer and re-mirrors the catalog in one call. Generic over the closure's error so it wraps both the `crate::Result` and `anyhow::Result` importers. Pairs with the existing `Published::decode(frames)` (frames in hand) as the byte-path equivalent. - Convert every `decode + sync` site to `decoding`: the Framed and Stream dispatchers, the TS H.264/H.265 streams, moq-rtc, moq-video, and moq-cli. - TS AAC: set the rendition `description` on the importer before `Published::new` (its attach-time mirror covers it) and route the jitter refinement through `decoding`. - `Published::sync` is now private: the only way to decode is through a path that syncs, so the footgun is gone. https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
`Producer::with_lenient_start` silently dropped non-keyframes that arrived before the first keyframe. Replace that implicit drop with an explicit `MissingKeyframe` error the producer returns, and let the one caller that wants to tolerate a mid-stream join (MPEG-TS) skip it. - New `container::MissingKeyframe` error: `Producer::write` returns it when a non-keyframe arrives with no open group (was a silent drop under lenient_start, or a generic ProtocolViolation without it). Wired into `crate::Error` and `fmp4::Error` via the `Container::Error` bound. - Drop `with_lenient_start` entirely. - Importers now surface MissingKeyframe for a pre-keyframe delta: h264 and h265 (and av1) write the delta through to the producer instead of pre-empting with a config error, erroring early only on an *unconfigurable keyframe* (NotInitialized / MissingSps / MissingSequenceHeader). vp8/vp9 already wrote straight through. - The TS importer wraps its H.264/H.265 decode in `skip_missing_keyframe`, so a capture that joins mid-GOP drops the leading deltas and resumes at the first keyframe (preserves survives_midstream_join + the dirty TS joins). https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Mirror the h264 split: separate the Annex-B parsing from the publisher. - New `codec::h265::Split`: a dumb Annex-B stream assembler that finds access-unit boundaries, caches VPS/SPS/PPS and re-inserts them ahead of each keyframe, and stamps wall-clock timestamps for stdin. No track, catalog, or codec config. `decode_stream` / `decode_frame` / `decode_from` / `seed` / `reset`, like h264. - `codec::h265::Import` now drives the splitter and owns the config: it scans the SPS the splitter packages into the first keyframe (or a seed buffer via `initialize`) to fill the catalog, errors on an unconfigurable keyframe, and writes pre-keyframe deltas through to the producer (which reports MissingKeyframe for a mid-stream join). It also implements `FrameDecode` so a caller with its own splitter can publish frames. - Adds the first H.265 unit tests (the splitter packaging path). https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Round out the streaming codecs: separate AV1 OBU parsing from the publisher, matching h264/h265. - New `codec::av1::Split`: a dumb OBU stream assembler that finds temporal-unit boundaries, flags keyframes (a sequence header or a KEY_FRAME), and stamps wall-clock timestamps for stdin. No track, catalog, or codec config. AV1 carries the sequence header inline ahead of keyframes, so unlike H.264/H.265 there's nothing to cache or re-insert; `seed` just prefixes leading metadata OBUs onto the next frame. The `ObuIterator` moves here. `decode_stream` keeps the per-OBU wall-clock timestamps. - `codec::av1::Import` now drives the splitter and owns the config: it scans the sequence header the splitter packages into the first keyframe (or an av1C / seed buffer via `initialize`) to fill the catalog, falls back to a minimal config on a parse failure, errors on an unconfigurable keyframe, and writes pre-keyframe deltas through to the producer (MissingKeyframe for a mid-stream join). It also implements `FrameDecode`. - Adds the first AV1 splitter unit tests (boundary + keyframe detection). https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
The lenient-start drop was replaced by the producer's MissingKeyframe; update the two h264 comments that still described the old behavior.
Resolved conflicts: - rs/moq-ffi/src/test.rs: kept dev's tokio::join! fix for the dynamic-track-request test (it supersedes this branch's earlier sequential subscribe_media fix). - rs/moq-mux/src/container/flv/import.rs: dev's new FLV importer used the removed with_lenient_start(); ported it to the MissingKeyframe model (drop the call, swallow MissingKeyframe at the video write so a mid-GOP join still works).
Review follow-up. There are no fixed tracks anymore, so a changed codec config just re-mirrors the catalog rendition instead of erroring. - Remove the `FixedTrackReconfigured` error variant from the h264, h265, and av1 error enums. - h264: drop `set_config`; avc1 (avcC) and avc3 (inline SPS) both resolve through one in-place `apply_config` that no-ops on an unchanged config. - h265 / av1: `configure_from_sps` / `apply_config` update the rendition in place on a change. - av1 `Split::decode_stream` / `decode_frame` take `impl Into<Option<Timestamp>>`, matching the h264 and h265 splitters. https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Review follow-up: a library crate shouldn't surface anyhow. Port the single-track codec importers that still used `anyhow::Result` to the crate's thiserror-based `crate::Result`, mirroring h264/h265/av1. - New `vp8::Error`, `vp9::Error`, and a `legacy::Error` (covering the MP2/AC-3/E-AC-3 header parsers), each wired into `crate::Error` via `#[from]` (`Vp8`/`Vp9`/`Legacy`). - `FrameHeader::parse` (vp8/vp9), the vp9 `BitReader`, the `ac3`/`eac3`/`mp2` `parse_header`s, and the vp8/vp9/legacy importer methods all return the typed errors now; no `anyhow::ensure!`/`bail!` remain in these modules. - Dropped the vp8/vp9 "fixed track cannot be reconfigured" bail too, so they update the rendition in place like the other codecs. The dispatcher test that asserted the old error now asserts in-place reconfiguration. - With every importer on `crate::Result`, `Published::decoding` drops its generic error parameter and just takes a `crate::Result` closure. https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
The importers owned a Split internally and exposed decode_frame/decode_stream; that duplicated the splitter and kept byte parsing in the publish layer. Move all byte parsing to dispatcher-owned splits so the importers only take frames. - `h264`/`h265`/`av1` `Import` lose their `split` field and the `decode_frame`/`decode_stream`/`decode_from` methods. They're pure publishers now: `decode(impl IntoIterator<Item = Frame>)` (FrameDecode) + config resolution (from the inline SPS/sequence-header in keyframes, or an avcC/av1C via `initialize`) + catalog + finish/seek/track. `initialize` resolves config without consuming the buffer; `seek` no longer resets a split. - `h264::Split` regains avc1: it's the sole h264 byte->frame engine for both wire shapes (framing + NALU length size only, config stays in the importer). `Mode`/`with_mode`/auto-detect moved here from the importer. - The Framed/Stream dispatchers, the TS container, moq-cli, moq-rtc, and moq-video now own a `Split` and drive `split.decode_X(buf) -> import.decode(frames)`. Small `build_h264`/`build_h265`/`build_av1` helpers in the dispatcher encode the "import reads config, split consumes" contract. 269 tests pass (incl. the byte-exact TS roundtrips and fMP4/MKV); the split gained avc1 unit tests. https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
A Split is for a raw byte stream (stdin / unknown boundaries); the
known-boundary decode_frame didn't belong on it.
- Rename `decode_stream` -> `decode` on all three splits, add `flush` (emit
the in-flight access/temporal unit), and drop `decode_frame`. `decode_from`
flushes at EOF. The Annex-B splits (h264/h265) keep a `tail` buffer so
`decode` fully consumes the caller's buffer (Framed's contract) while
retaining the trailing NAL across chunks; `flush` pulls it.
- Drop avc1 from `h264::Split` entirely. avc1 (length-prefixed + out-of-band
avcC) is not a stream and can't arrive over stdin, so `Mode`/`with_mode`/
`initialize`/`detect_mode` and the avc1 framing leave the splitter. avc1
becomes a free `h264::avc1_frame(data, length_size, pts)` helper.
- Framed splits its H264 arm into `Avc1 { length_size, import }` (no split,
wraps each AU via `avc1_frame`) and `Avc3 { split, import }`. Known-boundary
callers (Framed avc3/hev1/av01, TS, moq-rtc, moq-video) do `decode + flush`
per unit; stdin callers (Stream, moq-cli) flush the tail at finish/EOF.
265 tests pass (incl. TS byte-exact + fMP4/MKV roundtrips, moq-rtc bitstream).
https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
No caller used the splitters' decode_from async helper (the container importers have their own). Remove it from h264/h265/av1 Split; a caller that wants to drive a reader loops decode() + flush() itself.
A Split is just a byte stream; a dedicated "here are out-of-band parameter sets" hook (seed) contradicts that. Remove it from all three splits. The dispatchers' init buffer (the codec header passed to Framed::new / Stream::initialize) is now fed to the splitter via `decode` as the leading bytes of the stream: the importer still reads it for the catalog config, and the splitter caches any inline SPS/PPS the same way it would mid-stream, so a "parameter sets once up front, then bare keyframes" encoder still produces self-contained keyframes. av1C (the out-of-band 0x81 config record) is not an OBU stream, so it stays config-only and isn't fed to the splitter. The two seed unit tests now exercise the same property through `decode` (leading params + a later bare IDR -> self-contained keyframe). 265 tests pass; dependents (moq-cli/rtc/video/ffi/libmoq) build. https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Merge the new `publish` module into `import` and rename `Published` to `import::Track`. The module is now a directory: `import/mod.rs` (the `Framed`/`Stream` format dispatchers) plus a private `import/track.rs` (the catalog-bridge `Track`, `Renditions`, `FrameDecode`, `unique_track`), re-exported flat. The `publish::Published` stutter is gone, and "import" already names the ingest direction (the mirror of `export`). Add `moq_net::TrackDemand`, a cloneable, weak-backed watch-only handle (`name`/`used`/`unused`/`closed`) obtained from `TrackProducer::demand()`. It can't publish or close the track and doesn't pin the group cache, so callers can gate on subscriber demand without holding a writable producer. `TrackProducer: Clone` is left intact for now. Encapsulate the high-level front door: `Framed` no longer hands out a `&TrackProducer`. The match is now a private `producer()` helper behind curated `name()` / `subscribe()` / `demand()` accessors. moq-ffi's `MediaProducer` holds a `TrackDemand` instead of a cloned producer. The low-level `Track` and codec importers keep their public `track()`, since their callers build the track themselves, so moq-video/boy/audio/rtc/cli need no changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reshape the import layer so each codec importer owns its catalog rendition and deals only in whole frames, and so concurrently-produced tracks share one timeline. Catalog: add `catalog::VideoTrack` / `catalog::AudioTrack`, scoped handles (via `Producer::video_track` / `audio_track`) that publish one importer's rendition and retire it on drop. This replaces the `import::Track` wrapper plus the `Renditions` / `FrameDecode` traits, which are deleted; the `Published`-style mirror is gone. Importers: every codec importer is now `Import<E: CatalogExt = ()>` built with a single `new(track, catalog, [config])` (the `TrackRequest` constructor and `from_track` are dropped; the on-demand path accepts the request at the call site). They expose `demand() -> TrackDemand` instead of handing out a `TrackProducer`, take whole frames as `&[u8]` (no more `Buf`), and the per-importer `decode_buf` / `pts` helpers collapse into one `decode`. `Framed::decode_frame` becomes `Framed::decode(&[u8], pts)`. Sync: the wall-clock fallback moves to a `Clock` owned by the shared `catalog::Producer`, so audio and video synthesizing timestamps anchor to the same epoch (`Producer::timestamp` / `VideoTrack`/`AudioTrack::timestamp`). Containers and consumers (ffi/cli/rtc/gst/video/boy) are updated to the new constructors and `demand()`. Follow-ups: split `Framed` into `Framed` + `FramedTrack`, and give the TS/MKV/FLV/fMP4 containers their own `Split` modules so they too deal in frames. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`import::Framed`/`import::Stream` each mixed single-codec tracks with multi-track containers, so `demand()`/`name()`/`new_with_track` had to fail at runtime for containers (the `MultipleTracks` error). Split them along the multiplicity axis instead, keeping the framed-vs-stream axis: - `Track` / `TrackStream`: one codec on one MoQ track. `demand()` / `name()` are infallible; `Track::from_track` replaces `new_with_track`. - `Container` / `ContainerStream`: a container that may publish several tracks. No single-track handles. Both wrap one private `ContainerImpl`. A format only appears in a `*Stream` enum if it can recover frame boundaries from a raw byte stream, so streamability is now expressed in the type. Formats are typed per axis: `TrackFormat`, `TrackStreamFormat`, and `ContainerFormat` (reused for the container stream until a non-streamable container, e.g. RTP, arrives). `Error::MultipleTracks` is gone. Callers that take an arbitrary format string (libmoq, moq-ffi stream) dispatch into a small track/container enum.
The byte-parsing entry points took `T: Buf + AsRef<[u8]>` purely so they could advance the caller's buffer past consumed bytes — but the codec splitters and the ts/mkv/flv containers already copy their input into an internal scratch and retain the partial tail there, so the bound bought nothing except a generic that can't be a trait object. Make every decode take `&[u8]`: - Splits (h264/h265/av1) and the four containers drop the generic. av1's splitter gains the internal `tail` the others already had (its ObuIterator used to leave the partial OBU in the caller's buffer); fmp4 gains an internal buffer + `drain`, mirroring mkv/flv, instead of advancing the caller's `Buf`. - The import layer (Track/TrackStream/Container/ContainerStream) takes `&[u8]`; ContainerStream::initialize is dropped (containers are self-describing — decode already covers it). - Callers stop threading their own buffers: moq-ffi's stream producer drops its BytesMut, moq-cli reuses+clears a read chunk, moq-rtc skips a per-frame copy, moq-srt/moq-video/moq-hls pass slices. moq-hls loses the `InitNotConsumed` check (the remainder is no longer observable; the is-initialized check still guards a bad init segment). The internal NAL/OBU iterators stay generic over the internal buffer.
fmp4/ts/mkv/flv each had an async `decode_from(reader)` that looped `read_buf` into a chunk and fed it to `decode`. Nothing called them, and now that `decode` takes `&[u8]` the loop is trivial for a caller to write inline. Remove them and the `tokio::io` imports they pulled in.
It only ever gated moq-hls, which used it to turn an init segment with no moov into an early error and to re-check before each media fragment. Both are redundant: an importer that never resolved its config errors on its own at the first frame it can't place (the codec imports return NotInitialized, fmp4 returns NoMoov), so the readiness probe added a method to every importer for nothing. Remove it from the codec and container imports and the stream dispatchers; moq-hls relies on the decode error instead and drops its two now-unused error variants. The vp8/vp9/fmp4 tests already assert on the catalog snapshot, so they keep their coverage without the probe.
TrackFormat / TrackStreamFormat / ContainerFormat were a thin string mapping that every caller immediately resolved into a `new()` call, and moq-ffi's uniffi surface is string-based anyway, so the typed format was pure internal indirection. Have each importer's `new`/`from_track` take `format: &str` and parse it inline, erroring on a format it doesn't handle. Streamability is now encoded entirely by the type: `TrackStream::new` accepts only the self-delimiting codecs (avc3/hev1/av01), and `Container::new` / `ContainerStream::new` keep separate match lists so a future non-streamable container (e.g. RTP) can be added to `Container` alone — the role the proposed `ContainerStreamFormat` would have played. The two classify-then-build callers (libmoq `media_ordered`, moq-ffi `publish_media_stream`) build the track importer first and fall through to a container on `UnknownFormat`; libmoq keeps its own `UnknownFormat` error when neither matches. moq-gst passes string literals.
`Track::new` (mint a unique track) and `Track::from_track` (use a given track) shared the entire codec dispatch; the only difference was who created the track. Keep just the given-track form and call it `new`, so the importer never owns track creation — the caller mints with `unique_track` when it wants an on-demand track. The catalog rendition is still registered lazily when the codec config resolves, so dropping the broadcast parameter loses nothing. Callers mint as needed: moq-gst and moq-ffi `publish_media` mint then construct; `publish_media_on_track` just passes its requested track. libmoq `media_ordered` now tries the container first so a codec format doesn't mint a stray track before being recognized. `TrackStream` keeps minting in `new` — it has no given-track caller, so there's nothing to collapse.
Add `BroadcastProducer::reserve_track(name) -> TrackRequest`: a producer-authored deferred track, the same shape as a consumer-driven `requested_track()` but initiated by the producer. The track is discoverable immediately; its `TrackInfo` (timescale) is set when the importer accepts it. `Track`/`TrackStream::new` now take a `TrackRequest` and accept it, which is the single place a codec-specific timescale would be chosen (today it's the legacy microsecond timescale for all of them). Callers that minted a track now reserve one: moq-gst, moq-ffi `publish_media` / `publish_media_stream`, and libmoq `media_ordered` (still container-first so a codec format doesn't reserve a stray track). moq-ffi's dynamic flow stops accepting the request eagerly: it hands back a `MoqTrackRequest` (name / accept / abort), and `publish_media_on_track` lets the importer accept it — which also fixes a latent bug where a requested media track was accepted untimed and then rejected timed frames with `TimestampMismatch`.
dev independently reworked the moq-ffi dynamic flow (MoqTrackInfo, Option<TrackInfo>/Option<Subscription> params, and a pending/active MoqTrackProducer that accepts a requested track on first use). Keep that design and adapt its importer calls to this branch's moq-mux API: - publish_media / publish_media_stream reserve a track and hand the request to Track::new / TrackStream::new (container-first for stream). - publish_media_on_track takes the still-pending request off the MoqTrackProducer (new take_pending) and lets the importer accept it. - MediaProducer tracks demand (not the producer); MediaStreamProducer drops its buffer since the codec/container importers buffer internally. Drops the branch's own MoqTrackRequest in favor of dev's pending MoqTrackProducer, which covers the same "accept on first use" need.
Rather than dev's pending/active MoqTrackProducer, mirror moq-net's split: requested_track() returns a MoqTrackRequest (wrapping moq_net::TrackRequest) that you accept() into a MoqTrackProducer for raw writes, hand to publish_media_on_track for media (the importer accepts it), or abort() to reject. MoqTrackProducer goes back to wrapping a plain TrackProducer. Keeps dev's MoqTrackInfo and the Option<TrackInfo>/Option<Subscription> params; accept() takes Option<MoqTrackInfo> to match TrackRequest::accept.
requested_track() now returns MoqTrackRequest (not MoqTrackProducer), and publish_media_on_track takes the request. Mirror that in the hand-written wrappers: add a Python TrackRequest (accept/abort) and a Kotlin TrackRequest alias, point requested_track/requestedTracks at it, and accept the request for raw writes in the dynamic-track test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The import half of the moq-mux refactor. It lets
moq-muxfill a single track on demand without aBroadcastProducer/CatalogProducer, separates each codec's byte parsing from its publisher, and removes the manualsync()footgun.This started as "decouple the importers from the broadcast catalog" and grew (same branch, by request) to land the per-format splitters,
Published-owns-decode, and pure-publisher importers that were originally deferred. Ports every single-track importer (opus, H.264, H.265, AV1, VP8, VP9, AAC, and the legacy MP2/AC-3/E-AC-3 importer) plus the dispatchers and the TS container that embeds them. All existing callers keep working.1. The catalog bridge (
publishmodule)Renditionstrait: an importer exposes thehang::Catalogit publishes.Published: mirrors an importer's renditions into acatalog::Producerand retires them on drop. Generic over the catalog extensionEso it can attach to a container's extended catalog.unique_track(broadcast, suffix): mints a legacy single-codec track athang::container::TIMESCALE.Each importer is
new(TrackRequest)(on-demand) /from_track(TrackProducer)(broadcast push / fixed track) with a localhang::Catalogand a lazycatalog()(eager for audio). Nocatalog::Producer, noDrophook. There is no "fixed track" concept: a changed codec config re-mirrors the rendition in place rather than erroring.2. Per-codec splitters + pure-publisher importers
Byte parsing and publishing are fully separated:
codec::h264::Split/h265::Split/av1::Split: dumb byte->frame engines. They find access-unit / temporal-unit boundaries, flag keyframes, and stamp wall-clock timestamps for stdin. They own no track, catalog, or config. h264/h265 cache SPS/PPS(/VPS) and re-insert them ahead of each keyframe; h264'sSplithandles both avc1 (length-prefixed) and avc3 (Annex-B) shapes; AV1 carries the sequence header inline.decode_stream(unknown boundaries) /decode_frame(one AU) /decode_from/seed/reset.codec::{h264,h265,av1}::Importare pure frame publishers: they take already-split frames viadecode(impl IntoIterator<Item = Frame>)(theFrameDecodetrait) and resolve the catalog config from the inline SPS/sequence-header in the first keyframe (or an out-of-band avcC/av1C viainitialize). A keyframe that can't be configured is an error.Splitand drivesplit.decode_X(buf) -> import.decode(frames).3. Published owns decode (the
sync()footgun is gone)FrameDecode+Published::decode(impl IntoIterator<Item = Frame>): the frames path; it syncs the catalog after decoding.Published::decoding(|inner| ...): the byte-path/edit wrapper (still used by VP8/VP9 and the TS jitter edit), which also syncs.Published::syncis private — the only way to decode is a path that syncs.4. lenient_start -> MissingKeyframe
The container
Producerno longer silently drops pre-keyframe frames. Writing a non-keyframe with no open group returnsMissingKeyframe; importers write deltas straight through, and the TS/FLV containers swallowMissingKeyframeso a mid-stream join resumes cleanly at the next keyframe.5. thiserror everywhere
The single-track importers no longer surface
anyhow: vp8/vp9/legacy (and the ac3/eac3/mp2 header parsers) getthiserrorErrorenums wired intocrate::Errorvia#[from], matching h264/h265/av1.TS container
H.264/H.265/AAC/legacy per-PID streams build through
Published+ a per-PIDSplit. AAC's synthesizeddescriptionis set on the importer before attach (soPublished::new's mirror covers it) and the audio-burstjitterrefinement goes throughdecoding. External behavior is unchanged — the byte-exact roundtrip tests guard it.Notes
dev: breaking changes tomoq-muxpublic APIs (newSplittypes,FrameDecode,Published::{decode,decoding}, privatesync, importers lose their byte methods, removedwith_lenient_start/FixedTrackReconfigured), built on dev-only primitives (TrackRequest,TrackInfo::with_timescale,Frame.duration)..awaitinrs/moq-ffi/src/test.rs(superseded by dev'stokio::join!fix after the merge).Test plan
cargo test -p moq-mux(269 pass): on-demand + broadcast paths, lazy video catalogs, eager audio,Publishedsync/retire-on-drop and auto-sync viadecode/decoding, the splitter packaging + boundary + keyframe-detection tests (h264 avc1/avc3, h265, av1), in-place reconfiguration,MissingKeyframeon a delta-before-keyframe, TS byte-exact roundtrips (incl. mid-stream / dirty joins), fMP4/MKV roundtrips.cargo clippy --all-targets -- -D warnings,cargo fmt --all --check,RUSTDOCFLAGS=-D warnings cargo docclean.moq-cli,moq-rtc,moq-video,hang,moq-boy,libmoq,moq-ffibuild.moq-gstnot built here (missinggstreamer-1.0system lib); consumes the unchangedFramedAPI.(Written by Claude)